IN THE CLAIMS 

Please amend the claims as follows: 
• CLAIMS : 

1. A system (100) for integrative analysis of intrinsic (10) and 
extrinsic (11) audio-visual data, the system comprising: 

an intrinsic content analyser, the intrinsic content analyser 
being communicatively connected to an audio-visual source, the 
intrinsic content analyser being adapted to search the audio-visual 
source for intrinsic data and being adapted to extract intrinsic 
data using an extraction algorithm, 

an extrinsic content analyser, the extrinsic content analyser 
being communicatively connected to an extrinsic information source, 
the extrinsic content analyser being adapted to search the 
extrinsic information source and being adapted to retrieve 
extrinsic data using a retrieval algorithm, 

wherein the intrinsic data and the extrinsic data are 
correlated, thereby providing a multisource data structure. 

2. A system according to claim 1, wherein the retrieval of the 
extrinsic data is based on the extracted intrinsic data. 

3. A system according to claim 1, wherein the extraction and/or 
retrieval algorithm (s) is/are provided by a module. 
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4. A system according to claim 1, wherein a query is provided by 
a user, the query being provided to the extraction algorithm and 

• wherein the intrinsic data is extracted in accordance with the 
query. 

5. A system according to claim 1, wherein a query is provided by 
a user, the query being provided to the retrieval algorithm and 
wherein the extrinsic data is retrieved in accordance with the 
query. 

6. A system according to claim 1, wherein a feature reflected in 
the intrinsic and extrinsic data include textual, audio and/or 
visual features . 

7. A system according to claim 1, wherein the audio-visual source 
is a film (101) and wherein the extracted data include textual 
(104) , audio and/or visual features (105, 106) . 

8. A system according to claim 1, wherein the extrinsic 
information source is connected to and may be accessed via the 
Internet (103) . 

9. A system according to claim 1, wherein the extrinsic 
information source is a film screenplay (102) . 
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10. A system according to claim 9, wherein the extrinsic content 
analyser include knowledge about screenplay grammar, and wherein 
the extrinsic data is retrieved based on information extracted from 
the screenplay by use of the screenplay grammar. 

11. A system according to any of the claims- 9 or 10 wherein the 
identification (5) of persons in a film is obtained by means of the 
screenplay. 

12. A system according to any of the claime 9 or 10 wherein a 
feature in a film is analysed based on information included in the 
screenplay. 

13. A system according to claim 1, wherein the correlation of the 
intrinsic and extrinsic data is time correlation (121) , thereby 
providing a multisource data structure where a feature reflected in 
the intrinsic data is time correlated to a feature reflected in the 
extrinsic data. 

14. A system according to claim 13, wherein the time correlation 
is obtained by an alignment of a dialogue (120) in the screenplay 
to the spoken text (104) in the film and thereby providing a 
timestamped transcript (121) of the film. 
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15. A system according to claim 14, wherein a speaker 
identification in the film is obtained from the timestamped 
transcript. 

16. A system according to claim 9, wherein the screenplay is 
compared with the spoken text in the film by means of a self- 
similarity matrix (30) . 

17. A system according to claim 1, wherein a high-level 
information structure (5-9) is generated in accordance with the 
multisource data structure. 

18. A system according to claim 17, wherein the high-level 
information structure may be stored on a storage medium. 

19. A system according to claim 17, wherein an updated high-level 
information structure is generated, the updated high-level 
information structure being an already existing high-level 
information structure which is updated in accordance with the 
multisource data structure. 

20. A system according to claim 1, wherein the retrieval algorithm 
is a dynamic retrieval algorithm adapted to dynamically update 
itself by including additional functionalities in accordance with 
retrieved extrinsic data. 



N:\UserPub1ic\LIBE\PHNL\nl0401 63 _PreIim_cIaims.doc 



5 



21. A system according to claim 20, wherein the additional 
functionalities is obtained by training the retrieval algorithm on 
a set of features from intrinsic data using labels obtained from 
the extrinsic data. 

22. A system according to claim 9 and 2 1 , wherein the training is 
performed using at least one screenplay. 

23. A system according to 1, wherein an automatic ground truth 
identification in a film is obtained based on the multisource data 
structure for use in benchmarking algorithms on audio-visual 
content . 

24. A system according to 1, wherein an automatic scene content 
understanding in a film is obtained based on the textual 
description in the screenplay and the audio-visual features from 
the film content. 

25. A system according to 1, wherein an automatic labelling in a 
film is obtained based on the multisource data structure. 

26. A method for integrative analysis of intrinsic and extrinsic 
audio-visual information, the method comprising the steps of: 

searching an audio-visual source for intrinsic data and 
extracting intrinsic data using an extraction algorithm, 
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searching an extrinsic information source and retrieving 
extrinsic data using a retrieval algorithm, 

correlating the intrinsic data and extrinsic data, thereby 
providing a multisource data structure. 

27. A method according to claim 26 further comprising the step of 
generating a high-level information structure in accordance with 
the multisource data structure. 

28. A method according to claim 26, wherein the extrinsic content 
analyser include knowledge about screenplay grammar, and wherein 
the extrinsic data is retrieved using information extracted from 
the screenplay by use of the screenplay grammar. 

29. A method according to claim 26, wherein the retrieval 
algorithm is updated by training the algorithm on a set of 
extrinsic data. 

30. Computer programme product enabling a computer to be 
programmed to perform the method according to claim 26. 

31. Storage medium carrying the computer programme product 
according to claim 30. 

32. Programmed computer enabled to perform the method according to 
claim 26. 
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